The present invention relates to a pattern recognition apparatus and a method to correctly recognize an image pattern by eliminating unnecessary pattern information from the image pattern.
First, as an example of the prior art, pattern recognition for a face image is explained. Recognition of a person using a face image is executed without phisical contact for the user in comparison with discrimination by fingerprint. As a result, the user""s psychological burden is small and it is applied to various areas ranging from human interface to security systems. In the prior art, face image discrimination methods are classified into two methods.
As a first method, the position, shape, and size of specific feature points such as the pupil, nostril, and mouth are calculated to generate a feature vector. This feature vector is matched with the dictionary vector of the object person previously registered to calculate a similarity. The object person of the dictionary vector whose similarity is highest is determined to be the person in question. This method is called the xe2x80x9cStructure analysis methodxe2x80x9d.
As a second method, the face image is normalized by a geometrical transformation method such as two dimensional affine transformation based on feature points such as the pupil, and/or nostril. This normalized image is compared with each dictionary normalized image previously registered to calculate the similarity. In the same way as the first method, the object person of the dictionary normalized image whose similarity is highest is determined to be the person in question. This method is called the xe2x80x9cpattern methodxe2x80x9d. In comparison with the first method, the discrimination ratio of the second method is higher.
The xe2x80x9cSubspace methodxe2x80x9d is representative of the pattern method, which is widely used in character recognition. In this case, the angle between the input vector and each dictionary subspace is calculated as the similarity degree, and a category of the dictionary subspace whose angle is minimum is determined as the category of the input vector. In comparison with the simple relative method, the subspace method includes diffuseness in the dictionary side and a supplemental ability for transformation of the input pattern.
Furthermore, a xe2x80x9cmutual subspace methodxe2x80x9d, including higher supplemental ability is well known, which is disclosed in Japanese patent application PH10-66663. The mutual subspace method is explained in detail because of a prerequisite of the present invention. In the mutual subspace method, both the input side and the dictionary side are represented as the subspace, and xe2x80x9ccos2 xcex8xe2x80x9d of angle xe2x80x9cxcex8xe2x80x9d between the input subspace and the dictionary subspace is regarded as the similarity degree. In this case, xe2x80x9ccos2 xcex8xe2x80x9d is represented as the following equation.                                           cos            2                    ⁢          θ                =                              SUP                                                            u                  ∈                  P                                ,                                  xe2x80x83                                ⁢                                  v                  ∈                  Q                                                                                                  "LeftDoubleBracketingBar"                    u                    "RightDoubleBracketingBar"                                    ≠                  0                                ,                                  xe2x80x83                                ⁢                                                      "LeftDoubleBracketingBar"                    v                    "RightDoubleBracketingBar"                                    ≠                  0                                                              ⁢                      xe2x80x83                    ⁢                                                    "LeftBracketingBar"                                  (                                      u                    ,                    v                                    )                                "RightBracketingBar"                            2                                                                        "LeftDoubleBracketingBar"                  u                  "RightDoubleBracketingBar"                                2                            ⁢                                                "LeftDoubleBracketingBar"                  v                  "RightDoubleBracketingBar"                                2                                                                        (        1        )            
The projection matrix to the input subspace P is represented as p, and the projection matrix to the dictionary subspace Q is represented as Q. In this case, cos2 xcex8 of angle xcex8 between subspaces P and Q is the eigenvalue of QPQ or PQP.
This eigenvalue problem is replaced by an eigenvalue problem of a small dimension number. Assume that the eigenvector of QPQ is vxcex5Q. xcfx86, xcfx86 is respectively the base vector of subspace P, Q.                     P        =                              ∑                          m              =              1                        M                    ⁢                      ⟨                                          φ                m                            ,                              φ                m                                      ⟩                                              (        2        )                                Q        =                              ∑                          n              =              1                        N                    ⁢                      ⟨                                          φ                n                            ,                              φ                n                                      ⟩                                              (        3        )            
The eigenvalue problem of QPQ is represented as the following equation.
QPQ=xcexvxe2x80x83xe2x80x83(4)
                    v        =                              ∑                          k              =              1                        N                    ⁢                                    C              k                        ⁢                                          φ                k                            ⁡                              (                                  v                  ∈                  Q                                )                                                                        (        5        )            
The equation (5) is substituted to the right side of the equation (4).                               λ          ⁢                      xe2x80x83                    ⁢          v                =                              ∑                          k              =              1                        N                    ⁢                      φ            ⁢                          xe2x80x83                        ⁢            λ            ⁢                          xe2x80x83                        ⁢                          C              k                        ⁢                          φ              k                                                          (        6        )            
On the other hand, the equations (2) (3) are substituted to left side of the equation (4).                     QPQ        =                              ∑                          K              =              1                        N                    ⁢                                    ∑                              i                =                1                            N                        ⁢                                          ∑                                  m                  =                  1                                M                            ⁢                                                ∑                                      n                    =                    1                                    N                                ⁢                                                      ⟨                                                                  φ                        i                                            ,                                              φ                        i                                                              ⟩                                    ⁢                                      xe2x80x83                                    ⁢                                      ⟨                                                                  φ                        m                                            ,                                              φ                        m                                                              ⟩                                    ⁢                                      xe2x80x83                                    ⁢                                      ⟨                                                                  φ                        n                                            ,                                              φ                        n                                                              ⟩                                    ⁢                                      xe2x80x83                                    ⁢                                      C                    k                                    ⁢                                      φ                    k                                                                                                          (        7        )            
The equation (7) is arranged by changing the order of calculation.                     QPQ        =                              ∑                          K              =              1                        N                    ⁢                                    ∑                              m                =                1                            M                        ⁢                                          ∑                                  n                  =                  1                                N                            ⁢                                                (                                                            φ                      k                                        ,                                          φ                      m                                                        )                                ⁢                                  (                                                            φ                      m                                        ,                                          φ                      n                                                        )                                ⁢                                  C                  n                                ⁢                                  φ                  k                                                                                        (        8        )            
In the equations (6) (8), the same parameter xcfx86 k is aimed as follows.                               λ          ⁢                      xe2x80x83                    ⁢          Ck                =                              ∑                          m              =              1                        M                    ⁢                                    ∑                              n                =                1                            N                        ⁢                                          (                                                      φ                    k                                    ,                                      φ                    m                                                  )                            ⁢                              (                                                      φ                    m                                    ,                                      φ                    n                                                  )                            ⁢                              C                n                                                                        (        9        )            
In this case, assume the following replacement.
Ct=(C1, C2, . . . , CN)xe2x80x83xe2x80x83(10)
X=(Xij)xe2x80x83xe2x80x83(11)
                              X          ij                =                              ∑                          m              =              1                        M                    ⁢                                    (                                                φ                  i                                ,                                  φ                  m                                            )                        ⁢                          (                                                φ                  m                                ,                                  φ                  n                                            )                                                          (        12        )            
The equation (9) is regarded as the eigenvalue problem of matrix X as follows.
xcexc=Xcxe2x80x83xe2x80x83(13)
The maximum eigenvalue of X is represented as xe2x80x9ccos2 xcex81xe2x80x9d of minimum angle xcex81. The second eigenvalue is cos2 xcex82 of the angle along perpendicular direction of the maximum angle. In the same way, hereinafter, cos2 xcex8i (i=1, . . . , N) is calculated in order. The angle xcex8i (i=1, . . . , N) is well known as a canonical angle between two subspaces. In the mutual subspace method, the input side is represented as a subspace in the same way as the dictionary side. Therefore, supplemental ability of pattern transformation is very high. The input subspace is calculated by KL expansion for a plurality of input images. Alternatively, by using a simultaneous iterative method, the input subspace is generated from dynamic images in order.
In the face recognition method using images, discrimination ability is often affected by the following changes.
1. Influence of the change of illumination
2. Influence of time (change of hair, beard, wrinkles)
In short, it is important for the face recognition method to solve the above two problems.
(Problem 1)
As for the change of illumination of problem 1, face recognition as a three dimensional object is often affected by the change of illumination in comparison with the character recognition as a two dimensional object. In case outdoor light is irradiated to one side of the face and a shadow or a highlight occurs on the face, it is difficult for the computer to correctly recognize the face image as the person in question.
As mentioned-above, the mutual subspace method includes a high supplemental ability of pattern transformation, but a robust ability to influence the change of illumination is not improved. This is because a constraint is not assigned to the relation between two vectors u, v forming angle xcex8 calculated by the mutual subspace method. In short, the constraint is not assigned to a difference vector between two vectors u, v. Therefore, if the mutual subspace method is applied for the image including a change of illumination, the difference vector between the two vectors forming a minimum angle xcex8 includes the change element of the illumination. As for two different persons, the minimum angle between the two different persons is smaller than the true angle. Conversely, as for the same person, the minimum angle between the same person is larger than the true angle. As a result, the same person is often determined to be a different two persons.
(Problem 2)
As for the aging induced change of the problem 2, the discrimination ability decreases in the same way as the problem 1. In this case, because the difference vector includes pattern changes such as wrinkles that emerge over time, the discrimination ability often decreases.
As mentioned-above, the reason for the instability of these changes is that the discrimination is executed for the face image including pattern changes caused by the change of illumination or passage of years. Therefore, it is important to exclude from the image the pattern change elements that are unnecessary for the discrimination.
It is an object of the present invention to provide a pattern recognition apparatus and a method to correctly recognize the pattern by excluding unnecessary pattern change elements from the image.
According to the present invention, there is provided a pattern recognition apparatus, comprising: an input means for inputting an image pattern of the object to be recognized; an input subspace calculation means for calculating an input subspace from the image pattern; a dictionary subspace calculation means for calculating a dictionary subspace from a learning pattern of each object; a constraint subspace calculation means for calculating a constraint subspace from a plurality of input subspaces previously calculated according to constraints to suppress patterns unnecessary for recognition; a projection means for projecting the input subspace and the dictionary subspace onto the constraint subspace; and a recognition means for recognizing the object by comparing the projected input subspace with the projected dictionary subspace.
Further in accordance with the present invention, there is also provided a pattern recognition method, comprising the steps of: inputting an image pattern of the object to be recognized; calculating an input subspace from the image pattern; calculating a dictionary subspace from a leaning pattern of each object; calculating a constraint subspace from a plurality of input subspaces previously calculated according to constraints to suppress patterns unnecessary for recognition; projecting the input subspace and the dictionary subspace onto the constraint subspace; and recognizing the object by comparing the projected input subspace with the projected dictionary space.
Further in accordance with the present invention, there is also provided a computer readable memory containing computer readable instructions, comprising: an instruction means for causing a computer to input an image pattern of the object to be recognized; an instruction means for causing a computer to calculate an input subspace from the image pattern; an instruction means for causing a computer to calculate a dictionary subspace from a leaning pattern of each object; an instruction means for causing a computer to calculate a constraint subspace from a plurality of input subspaces previously calculated according to constraints to suppress patterns unnecessary for recognition; an instruction means for causing a computer to project the input subspace and the dictionary subspace onto the constraint subspace; and an instruction means for causing a computer to recognize the object by comparing the projected input subspace with the projected dictionary space.